Information processing apparatus, information processing method, and storage medium

ABSTRACT

An information processing apparatus includes an image obtaining unit configured to obtain an image, a first determining unit configured to determine a first image range to be used in making a determination related to inspection of an inspection target included in the image, based on a detection result of the inspection target from the image, and a second determining unit configured to determine a second image range to be used in recording an inspection result of the inspection target, the second image range being an image range indicating a wider range than a range indicated by the first image range.

BACKGROUND Field of the Disclosure

The present disclosure relates to an information processing apparatus,an information processing method, and a storage medium.

Description of the Related Art

Inspection of an infrastructural structure conventionally includesvisually determining damage degrees of deformed spots, such as a crackin a concrete surface, and manually compiling determination results andimages of the corresponding spots as a record of inspection results, ora report. To improve the efficiency of such inspection, Japanese PatentApplication Laid-Open No. 2018-122995 discusses a method for detecting adeformation from a captured image of an object to be inspected, andautomatically determining the damage degree based on the deformation.

As described above, the inspection of an infrastructural structureincludes recording not only the determination results but also theimages of the corresponding spots. However, the foregoing methoddiscussed in Japanese Patent Application Laid-Open No. 2018-122995 isintended to support the operation for determining the damage degree, andan image range extracted as the detection result of a deformation to beused in determining the damage degree may be unsuitable as the image tobe recorded in the report. An example of the image range used fordetermination is a local area, such as a deformation or a specificmember. If a bridge or an inner wall of a tunnel is inspected, it isdifficult from a report in which such a local image range is recorded tofind out the position of the spot in the structure or a global statearound the target. In the inspection of an infrastructural structure, afurther improvement in efficiency is therefore desired of the method fordetermining the image range for recording an inspection result.

SUMMARY

Some embodiments of the present disclosure are directed to improving theefficiency of an image-based operation by determining an appropriateimage range.

According to an aspect of the present disclosure, an informationprocessing apparatus includes an image obtaining unit configured toobtain an image, a first determining unit configured to determine afirst image range to be used in making a determination related toinspection of an inspection target included in the image, based on adetection result of the inspection target from the image, and a seconddetermining unit configured to determine a second image range to be usedin recording an inspection result of the inspection target, the secondimage range being an image range indicating a wider range than a rangeindicated by the first image range.

Further features of various embodiments will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a hardware configuration of aninformation processing apparatus according to a first exemplaryembodiment.

FIG. 2 is a block diagram illustrating a functional configuration of theinformation processing apparatus according to the first exemplaryembodiment.

FIG. 3 is a diagram for describing processing by the informationprocessing apparatus according to the first exemplary embodiment.

FIGS. 4A and 4B are flowcharts illustrating the processing by theinformation processing apparatus according to the first exemplaryembodiment.

FIGS. 5A, 5B, 5C, and 5D are diagrams for describing display controlprocessing according to the first exemplary embodiment.

FIGS. 6A, 6B, and 6C are diagrams for describing correction processingaccording to the first exemplary embodiment.

FIGS. 7A, 7B, and 7C are diagrams for describing examples of aninspection target.

FIG. 8 is a diagram for describing processing by an informationprocessing apparatus according to a second exemplary embodiment.

FIG. 9 is a flowchart illustrating processing by the informationprocessing apparatus according to the second exemplary embodiment.

FIG. 10 is a diagram for describing the processing by the informationprocessing apparatus according to the second exemplary embodiment.

FIG. 11 is a block diagram illustrating a functional configuration of aninformation processing apparatus according to a third exemplaryembodiment.

FIG. 12 is a flowchart illustrating processing by the informationprocessing apparatus according to the third exemplary embodiment.

FIGS. 13A and 13B are diagrams for describing the processing by theinformation processing apparatus according to the third exemplaryembodiment.

FIG. 14 is a diagram for describing processing by an informationprocessing apparatus according to a fourth exemplary embodiment.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments will be described below with reference to thedrawings.

In a first exemplary embodiment, an example of application to inspectionof an infrastructural structure will be described. The present exemplaryembodiment is not limited to the inspection of an infrastructuralstructure and can be applied to various image-based operations. First,the inspection of an infrastructural structure will be described.Examples of the infrastructural structure to be inspected include abridge, a tunnel, and a building. Hereinafter, infrastructuralstructures will be referred to simply as structures. The structures aredamaged over time from various causes, such as earthquakes and saltdamages. As damages develop, various deformations, such as cracks andprecipitates, appear in/on the surface of the structure, and the damageto the structure can thus be checked based on information about thesurface deformations. In the inspection of an infrastructural structure,the degrees of damage, or damage degrees, of areas or members aredetermined based on the state of corresponding deformations. Inconventional inspection, an inspector captures an image of a spotincluding a deformation or a specific member, visually determines thedamage degree, and records the determination result and the capturedimage together in a report. The damage degree is determined in terms ofranks “A”, “B”, “C”, and “D” according to evaluation grades based oncharacteristics of the deformation. While the damage degree will bedescribed to be ranked in four grades, this is not restrictive.

Hereinafter, the deformation or specific member in/on the surface of astructure to be subjected to a damage degree determination as a test inan inspection operation will be referred to as an inspection target. Theinspection target is an example of an object. In the present exemplaryembodiment, a method for appropriately determining both an image rangeto be used in making the damage degree determination of an inspectiontarget and an image range for recording the inspection target in areport for a captured image of the surface of a structure to beinspected will be described. The item to be determined is not limited tothe damage degree, and other items, such as the state of an object andthe likelihood of being an object, may be determined.

In the following description, the image range to be used in making thedamage degree determination will be referred to as a determination range(corresponding to a first image range). The image range for recordingthe inspection target in a report will be referred to as a report range(corresponding to a second image range). An information processingapparatus 100 according to the first exemplary embodiment determines adetermination range and a report range that include the same inspectiontarget and have respective different sizes. Specifically, theinformation processing apparatus 100 may determine an image rangesuitable for the damage degree determination as a determination range,and then determine a report range wider than the determination range.For example, the determination range may be a range including aninspection target with the inspection target at the center. Thisfacilitates finding out the position of the inspection target and thestate of the surroundings when the report is referred to afterward.Moreover, the information processing apparatus 100 may determine thereport range based on a predetermined characteristic portion(hereinafter, referred to as a landmark) from which the position of thereport range on the structure can be identified or estimated.Specifically, the information processing apparatus 100 determines thereport range so that the landmark is included. This makes finding outthe position of the inspection target even easier. Details of the firstexemplary embodiment will be described in detail below with reference toFIGS. 1 to 7C.

First, a configuration of the information processing apparatus 100according to the first exemplary embodiment will be described withreference to FIGS. 1 and 2 . FIG. 1 is a hardware configuration diagramof the information processing apparatus 100 according to the firstexemplary embodiment. As illustrated in FIG. 1 , the informationprocessing apparatus 100 includes a central processing unit (CPU) 101, aread-only memory (ROM) 102, a random access memory (RAM) 103, a harddisk drive (HDD) 104, a display unit 105, an operation unit 106, acommunication unit 107, and a system bus 108 connecting thesecomponents. The CPU 101 controls the entire information processingapparatus 100. The ROM 102 is a program memory and stores controlprograms for the CPU 101. The RAM 103 is used as a temporary storagearea, such as a main memory and a work area of the CPU 101.

The HDD 104 stores data and programs for use in processing to bedescribed below. The information processing apparatus 100 may include anexternal storage device instead of or in addition to the HDD 104. Forexample, the external storage device can be implemented by a medium(recording medium) and an external storage drive for accessing themedium. Known examples of such a medium include a flexible disk (FD), acompact disc read-only memory (CD-ROM), a digital versatile disc (DVD),a Universal Serial Bus (USB) memory, a magneto-optical (MO) disc, and aflash memory. The external storage device may be a network-connectedserver apparatus.

The display unit 105 is a device that outputs an image on a displayscreen. Examples thereof include a cathode-ray tube (CRT) display and aliquid crystal display. The display unit 105 may be an external deviceconnected to the information processing apparatus 100 in a wired orwireless manner. The operation unit 106 includes a keyboard and a mouse,and accepts various operations made by a user. The communication unit107 performs wired or wireless bidirectional communication with anexternal apparatus, such as another information processing apparatus, acommunication device, and a server apparatus, using conventionalcommunication techniques.

FIG. 2 is a block diagram illustrating a functional configuration of theinformation processing apparatus 100 according to the first exemplaryembodiment. The information processing apparatus 100 functions asfunctional units illustrated in FIG. 2 by the CPU 101 loading programsstored in the ROM 102 into the RAM 103 and executing the programs. Asanother example, various functions and processes of the informationprocessing apparatus 100 may be implemented by the CPU 101 loadingprograms from an external storage device connected to the informationprocessing apparatus 100. As yet another example, part of the functionalconfiguration of the information processing apparatus 100 may beimplemented by using a hardware circuit.

As illustrated in FIG. 2 , the information processing apparatus 100includes an image obtaining unit 200, an inspection target dataobtaining unit 210, a determination range determining unit 220, adetermination unit 230, a landmark data obtaining unit 240, and a reportrange determining unit 250. The information processing apparatus 100further includes a display control unit 260, a correction unit 270, anda storage unit 280. Various processes performed by the functional unitsillustrated in FIG. 2 will be described below with reference to FIG. 3 .FIG. 3 is a diagram for describing a series of processes performed bythe information processing apparatus 100 according to the firstexemplary embodiment.

The image obtaining unit 200 obtains an image of the structure to beinspected. The image for the image obtaining unit 200 to obtain will nowbe described. Since the structure inspection includes inspecting finedeformations on a concrete wall surface, a high-definition image of theentire surface of the structure is to be used. In the present exemplaryembodiment, the image obtaining unit 200 sections the surface of thestructure into a plurality of cells, and obtains a plurality of cellimages captured cell by cell. The cell images are associated with adesign drawing of the structure. A method where the informationprocessing apparatus 100 sequentially obtains the cell images andperforms a series of processes to be described below will be described.In the present exemplary embodiment, an example of inspecting a floorslab of a bridge will be described. For that purpose, the imageobtaining unit 200 obtains captured images of the surface of the bridgefloor slab. Inspection targets are cracks in the structure.

A rectangle 310 in FIG. 3 represents the design drawing of the bridgefloor slab. A rectangle 311 represents an image of specific cells of thefloor slab. The rectangle 311 is divided into areas in a grid pattern,and the divided areas correspond to cell images. An image 320 is a cellimage corresponding to a divided area 312 in the rectangle 311. Curves322 and 323 in the image 320 represent cracks. Chalk lines 321 representchalk lines drawn on the floor slab by an inspector in the past, andindicate the number of the specific area of the floor slab.

The inspection target data obtaining unit 210 obtains inspection targetdata from the image. The inspection target data means position data onan inspection target, and is referred to in determining thedetermination range and the report range. In the first exemplaryembodiment, the inspection target data obtaining unit 210 obtains theinspection target data by using a crack detection model trained todetect a detection target (here, a crack) from an image. The crackdetection model is a trained model stored in the HDD 104, and isgenerated by being trained with a large number of pieces of trainingdata each including a pair of images, namely, an image of a concretewall surface and a ground truth image indicating a crack position orcrack positions in the image. The ground truth image has the same sizeas the corresponding image, and stores 1 in pixels corresponding topixels of the corresponding image where the detection target is, and 0in the other pixels. Models, including models to be described below, canbe trained by using any machine learning algorithm. For example, aneural network algorithm can be used. If the inspection target dataobtaining unit 210 applies the crack detection model to an image, alikelihood map is obtained. The likelihood map contains values close to1 in areas that are likely to be the detection target on the image andvalues close to 0 in other areas. An image obtained by binarizing thelikelihood map with a predetermined value as a threshold will bereferred to as a detection result. In other words, deformations andspecific members to be inspected are detected based on output withrespect to input of the image to the crack detection model. In such amanner, the inspection target data obtaining unit 210 obtains a crackdetection result from the image.

An image 330 in FIG. 3 is a crack detection result obtained by applyingthe crack detection model to the image 320. Areas 331 and 332 are areaswhere pixels detected to be a crack continue. Data thus obtained by amodel will be referred to as detection data. The crack detection datacorresponds to the foregoing inspection target data. The area 331 of theimage 330 corresponds to the curve 322 in the image 320. The area 332 ofthe image 330 corresponds to the curve 323 in the image 320.

The determination range determining unit 220 determines an image rangeto perform damage degree determination processing on an inspectiontarget in the image as a determination range. The position of thedetermination range is determined based on the inspection target data.More specifically, the determination range determining unit 220determines the determination range in the image to be used in making adetermination related to the inspection of the inspection targetincluded in the image based on the detection result of the inspectiontarget from the image. In the first exemplary embodiment, thedetermination range determining unit 220 determines the determinationrange based on image conditions for the images used to train a damagedegree determination model to be described below, which is used by thedetermination unit 230. Specifically, the determination rangedetermining unit 220 determines the determination range by using thesame conditions about the image size and the position of the detectiontarget in the image as with the images used for training. For example,the determination range determining unit 220 sets the size of the imagerange to a predetermined size used in training the damage degreedetermination model, and locates the image range so that the centerposition thereof coincides with the center of the inspection targetdata. In the example of FIG. 3 , the determination range determiningunit 220 calculates the center of gravity of the area 331 representingthe inspection target data, and determines a determination range 351 bydetermining the position of an image range of the predetermined size sothat the center of gravity comes to the center. An image 350 in FIG. 3is the same image as the image 320.

The determination unit 230 makes a determination on an image cut out tothe determination range. In the first exemplary embodiment, thedetermination unit 230 makes a damage degree determination using a modelfor determining a damage degree of an image (damage degree determinationmodel) that is trained in advance and stored in the HDD 104. In thefirst exemplary embodiment, the damage degree determination model istrained to output a damage degree based on the state of a crack orcracks in an image since the inspection target is the crack. Forexample, the damage degree is ranked in four grades, ranks “A” to “D”,and the damage degree determination model is trained so that the moresevere the damage of an inspection target in the image is, the rankcloser to “A” is output. To generate such a model, a large number ofimages of the predetermined size are prepared, and each image ismanually given a damage degree rank by visual observation.

For example, if the image includes dense cracks or a crack with waterleakage, the damage is considered to be severe, and the damage degree isdetermined to be rank “A”. If the image includes only thin cracks, thedamage is considered to be not so severe, and the damage degree isdetermined to be rank “D”. The determination unit 230 trains the damagedegree determination model by using a large number of pairs eachincluding an image of the predetermined size and the damage degree givento the image. Any machine learning algorithm may be used for thetraining, and an example thereof is a support-vector machine.Alternatively, an architecture including a convolutional neural network(CNN) and fully-connected (FC) layers may be used as a deeplearning-based method. In making the damage degree determination usingthe damage degree determination model trained thus, the determinationunit 230 inputs an image of the same size as that of the images used fortraining into the damage degree determination model. In other words, thedetermination unit 230 can simply use and input the image cut out to thedetermination range determined by the determination range determiningunit 220. As a result, a damage degree rank is obtained.

The landmark data obtaining unit 240 obtains information about alandmark from the image. As employed herein, the landmark refers toinformation from which the position in the structure can be identifiedor estimated. Examples thereof include a structural part characteristicto each portion of the structure, and an artificial mark. Here, thechalk lines 321 are used as the landmark. In the first exemplaryembodiment, the landmark data obtaining unit 240 obtains position dataon a landmark within a predetermined range of distance from theinspection target data.

First, the landmark data obtaining unit 240 sets a landmark search rangeto search for landmarks near the inspection target. For example, thelandmark data obtaining unit 240 sets a landmark search range 333 of apredetermined size so that the center of the landmark search range 333coincides with the center of gravity of the area 331 (inspection targetdata). Next, the landmark data obtaining unit 240 detects a landmarkfrom a range corresponding to the landmark search range 333 in the image320. In the first exemplary embodiment, the landmark data obtaining unit240 obtains landmark data by using a model that is trained in advance todetect a predetermined landmark by a similar method to the method forgenerating the foregoing crack detection model. The landmark data refersto position data on the landmark. In the first exemplary embodiment, amodel for detecting chalk lines as the predetermined landmark is appliedto the image of the landmark search range 333, and an image 340 isobtained as a detection result. An area 341 of the image 340 correspondsto the chalk lines 321 in the image 320.

The report range determining unit 250 determines an image rangedifferent from the determination range on the image. Specifically, thereport range determining unit 250 determines the report range based onthe landmark data. More specifically, the report range determining unit250 determines the report range so that the landmark data and theinspection target data are included. For example, the report rangedetermining unit 250 obtains the leftmost, rightmost, topmost, andbottommost coordinate points from combined data of the inspection targetdata and the landmark data, and determines the report range to includethe coordinate points. While, in the first exemplary embodiment, thereport range determining unit 250 determines the report range so thatthe entire landmark data is included, the report range may be determinedto include only part of the landmark data. For example, a landmark witha large area, such as a diagonal member, is useful in identifying theposition on the structure even if only part of the landmark is includedin the report range.

Alternatively, the report range determining unit 250 may determine arange that includes the inspection target and is wider than thedetermination range as the report range. In such a case, the landmarkdata obtaining unit 240 is not needed. Alternatively, if no landmarkdata is obtained by the landmark data obtaining unit 240, the reportrange determining unit 250 may determine a range wider than the setlandmark search range as the report range. For example, if the landmarksearch range has a size of 1000×1000 pixels, the report rangedetermining unit 250 may determine the size of the report range to begreater, for example, 2000×2000 pixels. As a result, an image of asomewhat wider field of view can be cut out as the report range tofacilitate finding out which part of the landmark has been captured. Thereport range determining unit 250 may determine the size of the reportrange based on the size of the determination range. For example, thereport range determining unit 250 may determine a size obtained byexpanding the periphery of the determination range by a predeterminedamount as the size of the report range.

To allow the user to check the determination result of the determinationunit 230 and the report range determined by the report range determiningunit 250, the display control unit 260 controls generation and output ofa check screen (FIGS. 5A, 5B, 5C, and 5D) to the display unit 105. Thecheck screen displays an image (hereinafter, referred to as a reportimage) cut out of the image 320 based on the report range. Thecorrection unit 270 accepts user's corrections to the determinationresult and the report range via the operation unit 106. The storage unit280 records a set of the determination result and the report image foreach inspection target onto the HDD 104 as an inspection result.

FIGS. 4A and 4B are flowcharts illustrating processing by theinformation processing apparatus 100 according to the first exemplaryembodiment. FIG. 4A is a flowchart illustrating an overall procedure ofthe processing by the information processing apparatus 100. FIG. 4B is aflowchart illustrating details of processing for determining a reportrange in FIG. 4A (S450). The processing of the flowcharts of FIGS. 4Aand 4B is implemented by the CPU 101 loading programs stored in the ROM102 into the RAM 103 and executing the programs.

First, in S410, the image obtaining unit 200 obtains an image to beinspected from the HDD 104 or an external apparatus connected via thecommunication unit 107. Here, the image 320 of FIG. 3 is obtained. Inthe first exemplary embodiment, the image obtaining unit 200 obtains animage associated with the design drawing. However, the image obtainingunit 200 may obtain an image not associated with the design drawing.Next, in S420, the inspection target data obtaining unit 210 obtainsinspection target data targeted for determination from the imageobtained in S410. In the first exemplary embodiment, the inspectiontarget data obtaining unit 210 obtains the inspection target data fromthe image 320 by performing detection processing using a model (here,the crack detection model). The areas 331 and 332 in FIG. 3 correspondto the inspection target data. Alternatively, the inspection target dataobtaining unit 210 may detect inspection targets based on manual inputsmade on the image 320 without the detection processing. In other words,the inspection target data obtaining unit 210 may obtain positioninformation about inspection targets specified by the user's manualinputs on the image 320. Hereinafter, processing for determining adetermination range and a report range for the inspection target datacorresponding to the area 331 will be described.

Then, in S430, the determination range determining unit 220 determinesthe determination range for the image obtained in S410. In the firstexemplary embodiment, the determination range determining unit 220determines the determination range based on the inspection target dataobtained in S420 and the image conditions about the images used by thedetermination unit 230 in training the model (here, the damage degreedetermination model). For example, the determination range determiningunit 220 sets the size of the image range to the predetermined size usedin training the damage degree determination model, and locates the imagerange so that the center position thereof coincides with the center ofthe inspection target data. Here, the determination range 351 of FIG. 3is determined for the area 331.

If the inspection target exceeds the image range of the predeterminedsize, some of the deformation information affecting the determinationcan be missing in the determination range and the damage degree can failto be correctly determined. In view of this, a plurality of damagedegree determination models trained with images of differentpredetermined sizes may be stored in the HDD 104 so that thedetermination range determining unit 220 can select one of thepredetermined sizes based on the size of the inspection target. In sucha case, the determination unit 230, in the next S440, makes adetermination by using a damage degree determination model applicable toan image of the predetermined size that has been selected. Moreover, ifthe inspection target data lies near an end of the image, thedetermination range can protrude from the image obtained in S410. Insuch a case, the image obtaining unit 200 may obtain the cell imageadjoining on the design drawing, and the determination range determiningunit 220 may determine the determination range across the cell images.In the first exemplary embodiment, the determination range determiningunit 220 determines the determination range based on the imageconditions about the images used by the determination unit 230 intraining the model. However, this is not restrictive. The determinationrange may be determined based on an image size or an aspect ratio set inadvance.

In S440, the determination unit 230 makes a determination on thedetermination range determined in S430. In the first exemplaryembodiment, the determination unit 230 obtains the damage degree rankoutput as a result of inputting the image of the determination rangedetermined by the determination range determining unit 220 into thedamage degree determination model. Suppose here that the determinationunit 230 inputs the image of the determination range 351 in FIG. 3 intothe damage degree determination model, and the damage degree rank “C” isoutput.

In S450, the landmark data obtaining unit 240 and the report rangedetermining unit 250 perform processing for determining a report rangefor the image obtained in S410. Details of the processing fordetermining the report range will be described with reference to FIG.4B. As employed herein, the landmark is an object serving as a mark foridentifying the position of the corresponding inspection target on thestructure. In other words, the landmark is a conspicuous area havingcharacteristics different from those of surrounding areas. First, inS451, the landmark data obtaining unit 240 sets a landmark search areafor the inspection target data obtained in S420. For example, thelandmark data obtaining unit 240 sets the landmark search area bysetting an image range of the predetermined size so that the centerposition of the image range coincides with the center of the inspectiontarget data. This enables a search for a landmark near the inspectiontarget data. In the example illustrated in FIG. 3 , the landmark searchrange 333 is set for the area 331 serving as the inspection target data.

While the landmark data obtaining unit 240 sets the landmark searchrange with reference to the inspection target data, this method is notrestrictive, and the entire image 320 may be set as the landmark searchrange. In such a case, the landmark data obtaining unit 240 obtains thelandmark data from the entire range of the image 320 in S452, and thereport range determining unit 250 determines the report range in S453 sothat the obtained landmark data and the inspection target data areincluded. Furthermore, the report range determining unit 250 may selecta landmark to be included in the report range based on a predeterminedcondition. If the inspection target and the landmark are located awayfrom each other, the report image can become extremely large. Thus, theinformation processing apparatus 100 may set the maximum size of thereport range in view of observability of the inspection target in thereport image and balance in appearance. The maximum size of the reportrange is set based on the size of the inspection target, the recordingsize of the report image, or the resolution of the image. The size ofthe inspection target includes not only the overall size of theinspection target but also the sizes of parts constituting theinspection target. The report range determining unit 250 limits the sizeof the report range to below the maximum size. If the inspection targetdata is located near an end of the image, the image obtaining unit 200may obtain the cell image adjoining on the design drawing, and thelandmark data obtaining unit 240 may set the landmark search rangeacross the cell images.

In S452, the landmark data obtaining unit 240 detects a landmark fromthe image of the landmark search range set in S451. In the firstexemplary embodiment, the landmark data obtaining unit 240 obtainslandmark data by detecting the landmark using a model. In the exampleillustrated in FIG. 3 , the area 341 is detected as the landmark datafrom the image of the landmark search range 333. In other words, in theexample illustrated in FIG. 3 , the chalk lines 321 are the conspicuousarea having characteristics different from those of the surroundingareas, and detected as the landmark. While, in the first exemplaryembodiment, the landmark data obtaining unit 240 obtains the landmarkdata by detecting a landmark from an image by using a model, theposition of the landmark may be determined based on position informationmanually set in advance without the detection. Specifically, informationindicating a portion suitable as a landmark (for example, acharacteristic portion such as a numeral written in chalk) is set inadvance. Then, the landmark data obtaining unit 240 may accept an inputoperation about the portion suitable as a landmark on the image, andobtain position information for which the input operation is accepted.This enables generation of a report image that suits the user'spreference by the processing of the report range determining unit 250.

In S453, the report range determining unit 250 determines the reportrange based on the inspection target data obtained in S420 and thelandmark data obtained in S452. Here, an image range including the area331 and the area 341 in FIG. 3 is determined as a report range 352. Inthis example, the chalk lines 321 in the image 320 corresponding to thearea 341 represent a number indicating the specific area of the floorslab as described above. Thus, inclusion of such information in thereport range 352 facilitates identifying which part of the floor slabthe report range 352 is when the report image is referred to afterward.While the chalk lines 321 represent the number indicating the specificarea of the floor slab, the chalk lines 321 may be a curve or a markinstead of the number. If no landmark is detected in S452, the reportrange determining unit 250 may determine an image range of thepredetermined size wider than the determination range as the reportrange. For example, if no landmark is detected, the report rangedetermining unit 250 may determine the landmark search range as thereport range, or a range wider than the landmark search range as thereport range. While the report range determining unit 250 is describedto set the image range around the inspection target so that theinspection target is included in the report range, this is notrestrictive. The report range determining unit 250 may determine animage range not including the inspection target as the report range. Insuch a case, the report may include both the image of the determinationrange and the image of the report range. The report range here may be aconspicuous area located near the inspection target and capable ofidentifying the inspection target, or an area located near theinspection target and including a deformation of interest. The resultingreport thus includes information from which the position of theinspection target in the structure can be identified and informationabout the state of the surroundings of the inspection target forimproved convenience. By the foregoing processing of S451 to S453, thereport range can be determined for the image. After S453, the processingproceeds to S460. If a plurality of pieces of inspection target data isobtained in S420, the information processing apparatus 100 performs theprocessing of S430 to S453 on the next piece of inspection target data.

With the report range determined in the processing of S450, then inS460, the display control unit 260 generates and outputs the checkscreen to allow the user to check the result of the damage degreedetermination made in S440 and the report range determined in S450. Inthe first exemplary embodiment, the display control unit 260 generatesthe check screen in a report format and displays the check screen on thedisplay unit 105. FIGS. 5A, 5B, 5C, and 5D are diagrams for describingdisplay control processing performed by the display control unit 260.FIG. 5A is a diagram illustrating an example of the check screen. A leftregion 512 of a check screen 510 in FIG. 5A displays description fieldsfor items to be included in the report (such as date and damage type). Aright region 513 displays the report image. A description field 511indicating the result of the damage degree determination (here, “C”) isdisplayed as one of the items.

While one report image is displayed in the right region 513 of the checkscreen 510, a plurality of report images may be displayed. In such acase, the report range determining unit 250 determines a second reportrange wider than a first report range aside from the first report range,with the report range determined by the method described above as thefirst report range. Then, the display control unit 260 outputs the firstreport range and the second report range along with the determinationresult. A check screen 520 in FIG. 5B displays an image 523 of the firstreport range and an image 521 of the second report range along with thedescription field 511 for the damage degree. The image 521 is the sameas the image 320. The image 523 is an enlarged image of a local areaincluding an inspection target 522 in the image 521. Recording aplurality of report images of different sizes can further facilitatelocation identification.

The display control unit 260 may display the determination range and thereport range on the same screen in a comparable manner as in a checkscreen 530 of FIG. 5C. The check screen 530 of FIG. 5C displays adetermination range 534 of an inspection target 531 and a report range533 of the inspection target 531 as superimposed on the image 350.Displaying the determination range and the report range on the samescreen enables checking the output from the viewpoint of whether thereport range is appropriate for the determination range.

Referring back to the description of FIG. 4A, in S470, the correctionunit 270 corrects the result of the damage degree determination and thereport range output in S460 based on user operations. A specificcorrection method will be described with reference to FIGS. 6A, 6B, and6C. FIGS. 6A, 6B, and 6C are diagrams for describing correctionprocessing performed by the correction unit 270.

<Method for Correcting Result of Damage Degree Determination>

First, a method for correcting the result of the damage degreedetermination will be described. If a correct button 515 displayed inthe description field 511 for the damage degree on the check screen 510is selected, the display control unit 260 switches display from thecheck screen 510 to a correction screen 610 of FIG. 6A. The correctionscreen 610 displays the image 320 obtained in S410 with thedetermination range 351 superimposed thereon, as well as a result 613 ofthe damage degree determination. If the user desires to directly correctthe determination result, the correction unit 270 corrects the result613 of the damage degree by having the user directly edit the result 613of the damage degree determination.

On the other hand, if the user desires to correct the determinationrange 351, the correction unit 270 corrects the determination range 351based on operations on the determination range 351. In the firstexemplary embodiment, the determination range 351 has the predeterminedsize, and the correction unit 270 corrects the position of thedetermination range 351 without changing the size. For example, the usermoves the position of the determination range 351 by hovering a mousepointer 611 over the determination range 351 and performing a dragoperation. If the user then selects a determination button 612, thedetermination unit 230 makes a damage degree determination again. Then,the display control unit 260 receives the result of the damage degreedetermination made again from the determination unit 230, and updatesthe result 613 of the damage degree determination. If the user thenselects an OK button 614, the correction is finalized, and the displaycontrol unit 260 switches display from the correction screen 610 to thecheck screen 510. The corrected damage degree is reflected on thedescription field 511 for the damage degree on the switched check screen510. The above is the method for correcting the result of the damagedegree determination.

<Methods for Correcting Report Range>

Next, methods for correcting the report range will be described. Here, amethod for directly editing the report range and a method for selectinga report range candidate from a plurality of report range candidateswill be described. First, as a first method, the method for directlyediting the report range will be described. If the user selects acorrect button 516 for the report image on the check screen 510, thedisplay control unit 260 switches display from the check screen 510 to acorrection screen 620 of FIG. 6B. The correction screen 620 displays theimage 320 with the report range 352 superimposed thereon. For example,the user hovers the mouse pointer 611 over an end point of the reportrange 352 and makes a drag operation in the direction of the arrow 622.The correction unit 270 modifies the report range 352 into an imagerange 621 accordingly.

Next, as a second method, the method for selecting a report rangecandidate from a plurality of report range candidates will be described.In this method, the report range determining unit 250 stores theplurality of report range candidates into the ROM 102, RAM 103, etc. inadvance (for example, at the point in time of S450). The correction unit270 makes the user select an appropriate report range candidate fromamong the report range candidates, and corrects the report range to theselected candidate. A check screen 630 of FIG. 6C displays an imagerange 631 and an image range 632 on the image 320 as report rangecandidates. The ends of the image range 631 are located at the leftmost,rightmost, topmost, and bottommost coordinates of the inspection targetdata and the landmark data. The image range 632 is obtained by expandingthe periphery of the image range 631 by a predetermined amount. If theuser selects the image range 632 with the mouse pointer 611, thecorrection unit 270 determines the image range 632 as the report range.Such processing for selecting the report range candidate from theplurality of report range candidates may be performed in the processingof S450.

If the report range is modified by either of the foregoing methods andthen an OK button 623 or 633 is selected, the modification is finalized,and the display control unit 260 switches display to the check screen510. The right region 513 of the switched check screen 510 displays thereport image of the modified report range. The above are the methods forcorrecting the report range.

In S480, the storage unit 280 stores the result of the damage degreedetermination and the report range as a set into the ROM 102, RAM 103,etc. In the first exemplary embodiment, the storage unit 280 stores thedata in the report format displayed on the check screen 510 if an acceptbutton 517 on the check screen 510 is selected. Then, the series ofprocesses of the flowchart ends. The data for the storage unit 280 tostore is not limited to data in the report format, and may be data in afile format as illustrated in FIG. 5D. FIG. 5D illustrates a data tablestoring position information about determination ranges and reportranges. The data table stores coordinate information about thedetermination range and coordinate information about the report rangefor each inspection target in the image. The coordinate information isposition information on the image, whereas position information on thedesign drawing may be used instead.

In the first exemplary embodiment described above, image rangesrespectively suitable for determination purposes and report purposes canbe determined for an image in making the damage degree determination onan inspection target during structure inspection. An image from whichthe position and state of the inspection target can be easily found outafterward can thereby be recorded and left as the report image, withdetermination precision for the inspection target ensured. In otherwords, the efficiency of structure inspection can be improved. While theprocessing procedure has been described with reference to the flowchartsof FIGS. 4A and 4B, the processing order may be changed as appropriate.For example, the processing for determining the determination range(S430) and the processing for making the damage degree determination(S440) may be executed after the execution of the processing fordetermining the report range (S450).

<Landmark>

While, in the foregoing description, the chalk lines are used as thelandmark, the landmark is not limited thereto, and any characteristicportion (predetermined characteristic portion) from which the locationof the spot including the inspection target can be identified orestimated may be used. In structure inspection, an artificial mark, amember of the structure, a boundary of a structural member, a crackoccurred during construction, and an artifact can be used as variationsof the landmark. Hereinafter, the landmark will be described in detail.

Examples of the artificial mark include a mark drawn on the concretesurface with ink, aside from a chalk line. Examples of the memberinclude a diagonal member, a beam, a bearing, an iron plate, and a bolt.If the member is included in the report image, the location of the spotcan be found out based on the position of the member on the designdrawing. Examples of the boundary of a structural member include aborder between the sky and the concrete wall surface, and a borderbetween the ground and the concrete wall surface. The inclusion of theboundary of a structural member in the report image is useful in findingout the location and state of the spot since the captured image is knownto include an end of the structure. Examples of the crack occurredduring construction include formwork marks and joints. The inclusion ofjoints and formwork marks in the report image facilitates finding outthe scale of the inspection target since joints and formwork marks areoften regularly spaced. Examples of the artifact include a fence,stairs, a catwalk, a wire net, a net, cord, a pipe, a light, an electricmessage board, and a panel. Wide range artifacts, such as a fence,stairs, a catwalk, a wire net, and a net, have a relatively large area,and the inclusion of part of such an artifact in the report imagefacilitates identifying the location of the spot. Moreover, narrow rangeartifacts, such as a cord, a pipe, a light, an electric message board,and a panel, are also useful in finding out the location of the spot aswith the foregoing members. For example, if a light is included in thereport image of part of a tunnel, the inspection target is found to belocated at a high position, such as the ceiling. Other landmarksavailable may include a white line on the road, s maintenance hole, anda guardrail. The inclusion of such landmarks in the report imagefacilitates finding out the location and state of the spot.

<Grouping of Inspection Targets>

In the foregoing description, the inspection target is a deformation(crack), and the inspection target data includes information about asingle deformation. However, the inspection target data may includeinformation about a plurality of deformations. FIGS. 7A and 7B arediagrams illustrating examples of an inspection target including aplurality of deformations. An image 710 in FIG. 7A is a diagramillustrating an inspection target including discontinuous cracks.Discontinuous cracks can extend out and be connected with each otherover time. Thus, the information processing apparatus 100 groupsdiscontinuous cracks 711 and 712 so that the cracks 711 and 712 can berecorded together in a report. As a grouping method, first, theinspection target data obtaining unit 210 selects a piece of data ofinterest from pieces of data on the detected deformations, and sets adeformation search range for searching for a deformation within apredetermined range around the data of interest. Then, the inspectiontarget data obtaining unit 210 groups data on other deformations, if anydata is detected in the deformation search range, into one group.

An image 720 in FIG. 7B is a diagram illustrating an inspection targetincluding a plurality of exposed reinforcing iron bars 721 located apartfrom and adjacent to each other. Such a plurality of exposed reinforcingiron bars 721 can also be handled as data of the same group by theforegoing method. If data on a plurality of deformations is thus groupedas the inspection target, the determination range determining unit 220and the report range determining unit 250 perform the processing fordetermining the determination range and the report range on the group.

The examples of FIGS. 7A and 7B have dealt with the inspection targetincluding a plurality of deformations. However, the inspection target isnot limited to deformations. The inspection target may be a group of oneor more members, one or more deformations, or both. In other words, theinformation processing apparatus 100 may group different types ofobjects into one group. For example, the damage degree of a portion of acertain member can be determined based on the member and deformationsaround the member. If, for example, the range of corrosion around themember is wide, the damage degree can be determined to be high. Toinclude deformations affecting the determination into the determinationrange, the information processing apparatus 100 then groups the memberand the deformations around the member into one group, and determines adetermination range for the group.

An image 730 in FIG. 7C is a diagram illustrating an example of aninspection target including a member and deformations. The image 730includes a bolt 731 and cracks 732 and 733. In such a case, theinspection target data obtaining unit 210 obtains the respective piecesof detection data using the crack detection model and a model trained todetect a specific member. Then, the inspection target data obtainingunit 210 sets the deformation search range around the detection data onthe member, and groups the detection data on the member and thedetection data on deformations included in the deformation search rangeinto one group. In such a manner, the information processing apparatus100 may group a plurality of objects into one group, and determine thedetermination range and the report range for the grouped objects.

If the inspection target includes a member, the determination rangedetermining unit 220 may determine the determination range so that thecenter of the image range coincides with the center of the member. Insuch a case, the determination unit 230 can easily ensure determinationprecision by training the damage degree determination model with imagesincluding the member at the center, and applying the damage degreedetermination model to the determination range including the member atthe center. Moreover, the information processing apparatus 100 mayexclude insignificant data (for example, data not contributing to thedamage degree determination) from the group. An example of the data tobe excluded is data on a deformation without severe damage (thin crack).Compared to a thick crack, a thin crack has a low degree of damage andcan be regarded as the insignificant data. As another example, data at arelatively large distance from a specific member can be regarded as theinsignificant data in making the damage degree determination on thespecific member. The exclusion of the insignificant data from the groupcan remove unneeded information from the determination range andfacilitate ensuring the determination precision.

In the foregoing first exemplary embodiment, the information processingapparatus 100 determines the determination range based on the inspectiontarget data and the predetermined size. In a second exemplaryembodiment, the predetermined size is not provided, and thedetermination range is determined based on a range including theinspection target. Specifically, the information processing apparatus100 detects grid pattern cracks from an image, and determines thedetermination range and the report range based on a range including thegrid pattern cracks. The grid pattern cracks are a deformation occurringin a structural member, such as a floor slab of a bridge. The gridpattern cracks are a deformation resulting from the occurrence of cracksin one direction of the structural member (direction orthogonal to theaxis of the structural member) due to drying shrinkage, followed by theoccurrence of cracks in a direction orthogonal to that of the originalcracks (direction of the axis of the structural member) due torepetitive application of load to that portion from vehicles. Such adeformation is particularly significant because a closed area formed bythe cracks can come off and fall down. An information processingapparatus 100 according to the second exemplary embodiment has a similarconfiguration to that of the information processing apparatus 100according to the first exemplary embodiment. Thus, similar components tothose of the first exemplary embodiment are denoted by the samereference numerals, and redundant descriptions thereof will be omitted.Details of the second exemplary embodiment will be described below withreference to FIGS. 8 to 10 .

FIG. 8 is a diagram for describing processing for obtaining data on gridpattern cracks. In the second exemplary embodiment, the inspectiontarget data obtaining unit 210 detects the grid pattern cracks from animage 810, and obtains a range including the grid pattern cracks asinspection target data. Specifically, the inspection target dataobtaining unit 210 obtains polyline data to be described below byprocessing detection data on cracks, and identifies the range includingthe grid pattern cracks by using the polyline data. Chalk lines 811 inthe image 810 represent a number written in chalk as in the example ofthe first exemplary embodiment.

Processing by the information processing apparatus 100 according to thesecond exemplary embodiment differs from that of the flowchart of FIG.4A mainly in the details of the processing of S420. The processing willbe described in detail with reference to FIG. 9 , and a description ofthe other processing will be omitted. FIG. 9 is a flowchart illustratingprocessing performed by the inspection target data obtaining unit 210according to the second exemplary embodiment. After completion of theprocessing of FIG. 9 , the processing of S430 and the subsequentoperations in FIG. 4A is performed with the range including the gridpattern cracks as the inspection target data. The processing of FIG. 9is implemented by the CPU 101 loading a program stored in the ROM 102into the RAM 103 and executing the program.

First, in S410, the image obtaining unit 200 obtains an image to beinspected. In S901, the inspection target data obtaining unit 210obtains polyline data as the detection data on cracks. The polyline datais data including position information about the cracks detected, andrepresent areas connecting adjacent pixels detected as the cracks byline segments. In the second exemplary embodiment, the inspection targetdata obtaining unit 210 obtains the polyline data by applying imageprocessing, such as line thinning processing and vectorizationprocessing, to the detection data on cracks. An image 820 in FIG. 8 ispolyline data obtained from the image 810.

In S902, the inspection target data obtaining unit 210 detects closedareas formed by the cracks by using the polyline data obtained in S901.Here, the inspection target data obtaining unit 210 uses seed fill, animage processing technique. Seed fill is a method for filling acontinuous area in an image (inside or outside a closed area) by fillinga pixel at a starting point and then repeating processing for filling apixel adjoining a filled pixel if the adjoining pixel is not a contourpixel, with a given pixel on the image as the starting point. In theexample of FIG. 8 , the inspection target data obtaining unit 210applies the technique to the image 820 and detects areas 831, 832, 833,834, and 835 as closed areas as illustrated in an image 830. While, inthe second exemplary embodiment, the image processing technique is usedas the method for detecting closed areas, a model trained to detectclosed areas may be prepared in advance, and the closed areas may bedetected by using the model.

In S903, the inspection target data obtaining unit 210 groups adjoiningclosed areas of the closed areas detected in S901. In the example ofFIG. 8 , the closed areas 831, 832, 833, and 834 adjoin each other andare thus grouped into a group 836. The closed area 835 has no adjoiningclosed area and is regarded as a group by itself. Not only the adjoiningclosed areas but also an area of polyline including the adjoining closedareas may be grouped into the same group. Then, in S904, the inspectiontarget data obtaining unit 210 selects one of the groups obtained inS903. In S905, the inspection target data obtaining unit 210 determineswhether the selected group includes grid pattern cracks. Specifically,the inspection target data obtaining unit 210 makes the determinationbased on the closed area(s) included in the group and/or the directionof the polyline. As an example, the inspection target data obtainingunit 210 determines whether there are two or more closed areas in thegroup. As another example, the inspection target data obtaining unit 210determines whether the polyline passing through the group includes twoor more polylines extending vertically and two or more polylinesextending horizontally with respect to the structural member. Thedirection of a polyline is detected, for example, from the direction ofa vector connecting the starting point and end point of the polyline.

If, as a result of the determination processing of S905, the group isdetermined to include the grid pattern cracks (YES in S906), theprocessing proceeds to S907. In S907, the inspection target dataobtaining unit 210 adds the group to the inspection target data. On theother hand, if the group is determined to not include the grid patterncracks (NO in S906), the processing proceeds to S908. In the example ofFIG. 8 , the group 836 is determined to be a range including the gridpattern cracks. On the other hand, the group of the closed area 835 isdetermined to not be the range including the grid pattern cracks. InS908, the inspection target data obtaining unit 210 checks whether thegrid pattern crack determination processing has been performed on allthe groups formed in S903. If all the groups have been processed (YES inS908), the processing of the flowchart of FIG. 9 ends, and theprocessing returns to the flowcharts of FIGS. 4A and 4B. On the otherhand, if there is a group on which the grid pattern crack determinationprocessing has not been performed (NO in S908), the processing returnsto S904 to select the next group.

By the foregoing processing of the flowchart of FIG. 9 , a rangeincluding grid pattern cracks (specific object) can be obtained as theinspection target data. The subsequent processing can be performed by asimilar method to that of the first exemplary embodiment. First, thedetermination range determining unit 220 determines a determinationrange 842 for an image 840 (the same as the image 810) so that the group836 (inspection target data) is included. Then, the determination unit230 makes a damage degree determination on the determination range 842,and obtains the result of the damage degree determination. Next, thereport range determining unit 250 determines the report range based onthe inspection target data and the landmark data as in the firstexemplary embodiment. Suppose here that the landmark data obtaining unit240 has obtained detection data (landmark data) on an area 843 as adetection result of the chalk lines 811, and the report rangedetermining unit 250 determines a report range 841 so that the group 836and the area 843 are included.

In the second exemplary embodiment, the determination unit 230 makes adamage degree determination using a damage degree determination modeltrained with determination images of grid patterns cracks.Alternatively, the determination unit 230 may calculate grid-to-griddistances and make the damage degree determination based on thegrid-to-grid distances without using the damage degree determinationmodel. Specifically, first, the determination unit 230 calculates acrack-to-crack distance of each closed area in the grid pattern cracks.The crack-to-crack distance is detected from the number of pixels andconverted into an actual size. The determination unit 230 furtherdetermines an average, a minimum value, or a maximum value of thecrack-to-crack distances of the closed areas, and uses the value as thegrid-to-grid distance of the grid pattern cracks. The determination unit230 compares the grid-to-grid distance with a predetermined criterionand makes the damage degree determination. For example, if thegrid-to-grid distance is 20 cm or less, the damage degree rank is A.

In the second exemplary embodiment described above, image rangesrespectively suitable for determination purposes and report purposes canbe determined for a range including grid pattern cracks serving as theinspection target in structure inspection. An image from which theposition and state of the inspection target can be easily found outafterward can thereby be recorded and left as the report image, withdetermination precision for the inspection target ensured. In otherwords, the efficiency of structure inspection can be improved.

While, in the foregoing description, the inspection target is gridpattern cracks, the inspection target may include not only grid patterncracks but also hexagonal pattern cracks and a closed crack area (closedcrack). The hexagonal pattern cracks refer to a plurality of cracksdeveloped to intersect with each other and form closed areas in ahexagonal pattern. The grid pattern cracks, the hexagonal patterncracks, and the closed crack are examples of the specific object. Todetect ranges including the grid pattern cracks, the hexagonal patterncracks, and the closed crack in a collective manner, the inspectiontarget data obtaining unit 210 detects all intersections of vectorsconstituting the polyline, and obtains the inspection target data basedon density of the intersections.

FIG. 10 is a diagram for describing processing for obtaining theinspection target data by using the intersections of polyline vectors.An image 1010 is the same image as the image 820, and illustratespolyline data. Points indicated by black dots represent vectorintersections. For example, the inspection target data obtaining unit210 sections the image 1010 into a plurality of cells, calculates thedensity of intersections in each cell, and determines the presence ofdeformation in a high density cell. Such a method involves lessprocessing and can reduce execution time, although the inspection targetdata does not have as high positional precision as in the processing ofthe flowchart of FIG. 9 . A method using the intersections and thevectors may be employed instead of the method using the density of theintersections. For example, the inspection target data obtaining unit210 traces the polyline vectors starting at an intersection asillustrated by a double-dotted dashed line 1011 in the image 1010, anddetermines that there is a closed area (deformation) if the vectors aretracked back to the original intersection. The method using theintersections and the vectors can improve the positional precision ofthe inspection target data compared to the method using the density ofthe intersections.

In a third exemplary embodiment, an information processing apparatus 100determines a report range by using a method different from those of theforegoing exemplary embodiments. In the foregoing exemplary embodiments,the information processing apparatus 100 detects chalk lines as alandmark. In the third exemplary embodiment, the information processingapparatus 100 detects a plurality of types of landmarks set in advance.If the plurality of types of landmarks is detected, the informationprocessing apparatus 100 selects one of the landmarks and includes theselected landmark into the report range. For example, a landmark morelikely to lead to location identification when the report image isobserved afterward can be selected to facilitate finding out where theinspection target is. In the third exemplary embodiment, landmark dataobtained by a landmark data obtaining unit 240 will be referred to ascandidate data. Landmark data selected to be included into the reportrange from among pieces of candidate data will be referred to asselection data. Details of the third exemplary embodiment will bedescribed in detail below with reference to FIGS. 11 to 13B.

FIG. 11 is a block diagram illustrating an example of a functionalconfiguration of the information processing apparatus 100 according tothe third exemplary embodiment. The information processing apparatus 100functions as functional units illustrated in FIG. 11 by a CPU 101loading programs stored in a ROM 102 into a RAM 103 and executing theprograms. The information processing apparatus 100 according to thethird exemplary embodiment further includes a priority setting unit 1110in addition to the functional units of the information processingapparatus 100 according to the first exemplary embodiment. The prioritysetting unit 1110 sets degrees of priority among the pieces of candidatedata obtained by the landmark data obtaining unit 240. The report rangedetermining unit 250 determines the selection data from among the piecesof candidate data based on the degrees of priority, and determines thereport range based on the selection data.

The processing by the information processing apparatus 100 according tothe third exemplary embodiment is similar to that of the flowchart ofFIG. 4A, but the processing where the report range determining unit 250determines the report range (S450) differs. Details of the processingfor determining the report range will now be described with reference toFIGS. 13A and 13B and the flowchart of FIG. 12 . The processing of theflowchart of FIG. 12 is implemented by the CPU 101 loading programsstored in the ROM 102 into the RAM 103 and executing the programs.

FIGS. 13A and 13B are diagrams for describing the processing fordetermining the report range by using the selection data. In the thirdexemplary embodiment, a method where the information processingapparatus 100 obtains an image 1310 of FIG. 13A and performs a series ofprocesses to be described below on the image 1310 will be described. Theimage 1310 includes a crack 1311, chalk lines 1312, and a wire net 1313.Suppose here that the information processing apparatus 100 has obtaineddetection data on the crack 1311 as inspection target data anddetermines a determination range and a report range based on theinspection target data.

First, in S1201, similar to S451 of FIG. 4B, the report rangedetermining unit 250 sets a landmark search range for the inspectiontarget data on the image 1310. FIG. 13A illustrates a landmark searchrange 1314 set for the detection data on the crack 1311. Next, in S1202,the landmark data obtaining unit 240 obtains candidate data on landmarksfrom the landmark search range 1314. Here, the landmark data obtainingunit 240 applies a model for detecting chalk lines and a model fordetecting artifacts, such as a wire net and a cord, to the image of thelandmark search range 1314. The landmark data obtaining unit 240 therebyobtains detection data corresponding to the chalk lines 1312 anddetection data corresponding to the wire net 1313. Such pieces ofdetection data are the candidate data on landmarks.

Then, in S1203, the landmark data obtaining unit 240 determines whetherthere is the candidate data obtained. If there is determined to be thecandidate data (YES in S1203), the processing proceeds to S1204. InS1204, the priority setting unit 1110 sets a degree of priority for eachpiece of candidate data. In S1205, the priority setting unit 1110determines the selection data from among the pieces of candidate databased on the set degrees of priority. On the other hand, if there isdetermined to be no candidate data (NO in S1203), the processingproceeds to S1206 since there is no landmark to set the degree ofpriority for in the landmark search range 1314.

The processing of S1204 will now be described in detail. The prioritysetting unit 1110 stores degrees of priority associated with the typesof landmarks in advance, and sets the degrees of priority correspondingto the types based on the types of candidate data. Alternatively, thepriority setting unit 1110 may set the degrees of priority for thecandidate data based on the result of an inquiry made of the user aboutthe degrees of priority in advance. For example, the priority settingunit 1110 displays a screen from which the order of priority amongvarious landmarks including chalk lines, members, block boundaries, andartifacts can be specified on the display unit 105, and sets the degreesof priority based on user operations. If the degree of priority of chalklines is set to be high here, the degrees of priority of the candidatedata (the detection data on the chalk lines 1312 and the detection dataon the wire net 1313) are set so that the detection data on the chalklines 1312 has a higher degree of priority than that of the detectiondata on the wire net 1313. In such a case, in S1205, the detection dataon the chalk lines 1312 is determined as the selection data since thedegree of priority of chalk lines is high. The number of pieces ofselection data is not limited to one, and a plurality of pieces ofcandidate data ranked higher in the order of priority may be determinedas the selection data. If there is obtained only one piece of candidatedata, the candidate data may be simply determined as the selection data.

In S1206, similar to S453 of FIG. 4B, the report range determining unit250 determines the report range based on the inspection target data andthe selection data. FIG. 13B illustrates a report range 1321 determinedto include the detection data on the crack 1311 and the detection dataon the chalk lines 1312 for an image 1320 that is the same as the image1310. After S1206, the processing proceeds to S460.

In the third exemplary embodiment described above, a landmark morelikely to lead to location identification can be included in the reportimage range by priority. This further facilitates finding out theposition of the inspection target when the report is observed afterward.In other words, the efficiency of the structure inspection can beimproved.

<Method for Setting Degrees of Priority>

The method for setting the degrees of priority is not limited to themethod described with reference to the flowchart of FIG. 12 . Possiblemethods include a method based on a positional relationship between theinspection target data and the candidate data, a method based on rarityof the candidate data, and a method based on conspicuity of thecandidate data. The methods will now be described. First, the methodbased on the positional relationship between the inspection target dataand the candidate data will be described. The information processingapparatus 100 calculates a distance between each piece of candidate dataand the inspection target data. The priority setting unit 1110 then setshigher degrees of priority for pieces of candidate data closer to theinspection target data.

Next, the method based on the rarity of the candidate data will bedescribed. In this method, the priority setting unit 1110 sets thedegrees of priority of rare landmarks on the structure to be high on theassumption that the use of landmarks smaller in number on the structureis more likely to lead to location identification. Specifically, first,the information processing apparatus 100 applies models suitable for therespective types of landmarks to each of a plurality of cell images tocalculate the total numbers of pieces of detection data corresponding tothe respective models. The smaller the total number, the higher therarity. Thus, the priority setting unit 1110 sets higher the degrees ofpriority for the types of candidate data with the smaller total numbers.For example, cracks occurring on a wall surface during construction(joints and formwork marks) are greater in number than diagonal members,and thus are less likely to lead to location identification whenobserved afterward. Landmarks more likely to lead to locationidentification can thus be preferentially included in the report imagerange by reducing the degrees of priority of joints and formwork marksand increasing the degree of priority of diagonal members.

Then, the method based on the conspicuity of the candidate data will bedescribed. In this method, more conspicuous landmarks are included intothe report range. Specifically, first, the information processingapparatus 100 calculates a luminance difference between inside andoutside of each piece of candidate data based on the pixel values withinthe candidate data and the pixel values around the candidate data. Thegreater the luminance difference, the higher the conspicuity. Then, thepriority setting unit 1110 sets higher degrees of priority for pieces ofcandidate data with larger luminance differences. The method based onthe conspicuity is not limited to luminance differences, and may use thesizes of the areas, for example.

In S1202 of FIG. 12 , the landmark data obtaining unit 240 obtains thecandidate data on landmarks by applying the models suitable for therespective types of landmarks. However, the method for obtaining thecandidate data is not limited thereto. For example, the candidate datamay be obtained by using a saliency map. The saliency map is a mapvisualizing areas estimated to more likely draw human attention in theimage. For example, the saliency map is obtained by generating maps ofcharacteristic spots detected from the image in terms of items such asluminance information and color information, and determining a linearsum of the maps pixel by pixel. The landmark data obtaining unit 240converts pixels below a predetermined value in the saliency map into 0,whereby areas where non-zero pixels continue are obtained as thecandidate data. Then, the landmark data obtaining unit 240 sets thedegrees of priority based on the pixel values constituting the candidatedata. For example, the landmark data obtaining unit 240 obtainsstatistics, such as an average and a maximum value, of the pixel valuesin each piece of candidate data, and sets the degrees of priority basedon the statistics.

In the foregoing exemplary embodiments, the information processingapparatus 100 determines the determination range and the report rangefrom a single image. By contrast, in a fourth exemplary embodiment, aninformation processing apparatus 100 uses two images including the sameinspection target to determine the determination range from one of thetwo images and the report image from the other. The image to applydamage degree determination and the image to be recorded for reportpurposes can thus be captured by respective appropriate methods. Sincethe fourth exemplary embodiment can be implemented by a similarconfiguration to that of the first exemplary embodiment, redundantdescriptions thereof will be omitted. The inspection target in thefourth exemplary embodiment is grid pattern cracks. Differences of theimage obtaining unit 200, the determination range determining unit 220,and the report range determining unit 250 from those of the firstexemplary embodiment will now be described with reference to FIG. 14 .

The image obtaining unit 200 obtains a plurality of images including thesame inspection target. Specifically, the image obtaining unit 200obtains an image 1410 of FIG. 14 as the image to apply the damage degreedetermination. The image 1410 is an image of a wall surface of astructure, captured from right in front thereof to show the inspectiontarget in an enlarged scale. Meanwhile, the image obtaining unit 200obtains an image 1420 of FIG. 14 as an image to cut out the reportrange. The image 1420 is an image of the wall surface of the structure,captured so that surroundings of the inspection target are alsoincluded. The images 1410 and 1420 include the same inspection target.Areas 1421 and 1422 of the image 1420 include beams. The fourthexemplary embodiment uses the beams as the landmarks.

Next, processing by the determination range determining unit 220 and thereport range determining unit 250 will be described. First, thedetermination range determining unit 220 determines a determinationrange 1431 for the image 1410 so that the inspection target is included.Then, the report range determining unit 250 determines a report range1441 for the image 1420 so that the inspection target and part of thelandmarks (beams) are included. Since both of the processes can beperformed by the method described in the first exemplary embodiment, adetailed description thereof will be omitted.

While the method using two images including the same inspection targethas been described above, the information processing apparatus 100 mayuse a viewpoint-converted image as at least one of the two images. Insuch a case, the image obtaining unit 200 generates theviewpoint-converted image where the inspection target is seen from avirtual viewpoint, based on images of the same inspection targetcaptured at a plurality of positions. Then, at least one of thedetermination range determining unit 220 and the report rangedetermining unit 250 performs processing using the viewpoint-convertedimage.

In the fourth exemplary embodiment described above, images respectivelyappropriate for determination purposes and report purposes can be cutout by determining the determination range and the report range using aplurality of images including the same inspection target. An image fromwhich the position and state of the inspection target can be easilyfound out afterward can thereby be recorded and left as the reportimage, with determination precision for the inspection target ensured.In other words, the efficiency of structure inspection can be improved.

While the present disclosure has described above the exemplaryembodiments thereof, the foregoing exemplary embodiments are merelyexamples of embodiments, and the technical scope of every embodimentshould not be interpreted as limited thereto. In other words, theexemplary embodiments can be practiced in various forms withoutdeparting from the technical concept or main features of the same.

Some exemplary embodiments can be implemented by processing forsupplying a program for implementing one or more functions of theforegoing exemplary embodiments to a system or an apparatus via anetwork or a storage medium, and reading and executing the program byone or more processors in a computer of the system or apparatus. Acircuit for implementing one or more functions (for example, applicationspecific integrated circuit (ASIC)) may be used for implementation.

According to the foregoing exemplary embodiments, the efficiency of animage-based operation can be improved by appropriately determining imageranges.

Other Embodiments

Some embodiments can also be realized by a computer of a system orapparatus that reads out and executes computer-executable instructions(e.g., one or more programs) recorded on a storage medium (which mayalso be referred to more fully as a ‘non-transitory computer-readablestorage medium’) to perform the functions of one or more of theabove-described embodiment(s) and/or that includes one or more circuits(e.g., application specific integrated circuit (ASIC)) for performingthe functions of one or more of the above-described embodiment(s), andby a method performed by the computer of the system or apparatus by, forexample, reading out and executing the computer-executable instructionsfrom the storage medium to perform the functions of one or more of theabove-described embodiment(s) and/or controlling the one or morecircuits to perform the functions of one or more of the above-describedembodiment(s). The computer may comprise one or more processors (e.g.,central processing unit (CPU), micro processing unit (MPU)) and mayinclude a network of separate computers or separate processors to readout and execute the computer-executable instructions. Thecomputer-executable instructions may be provided to the computer, forexample, from a network or the storage medium. The storage medium mayinclude, for example, one or more of a hard disk, a random-access memory(RAM), a read only memory (ROM), a storage of distributed computingsystems, an optical disk (such as a compact disc (CD), digital versatiledisc (DVD), or Blu-ray Disc (BD)™), a flash memory device, a memorycard, and the like.

While the present disclosure has described exemplary embodiments, it isto be understood that some embodiments are not limited to the disclosedexemplary embodiments. The scope of the following claims is to beaccorded the broadest interpretation so as to encompass all suchmodifications and equivalent structures and functions.

This application claims priority to Japanese Patent Application No.2020-171113, which was filed on Oct. 9, 2020 and which is herebyincorporated by reference herein in its entirety.

What is claimed is:
 1. An information processing apparatus comprising:one or more computer-readable storage media; and one or more processorsthat are in communication with the one or more non-transitorycomputer-readable storage media, wherein the one or more processors andthe one or more non-transitory computer-readable storage media areconfigured to: obtain an image; determine a first image range to be usedin making a determination related to inspection of an inspection targetincluded in the image, based on a detection result of the inspectiontarget from the image; and determine a second image range to be used inrecording an inspection result of the inspection target, the secondimage range being an image range indicating a wider range than a rangeindicated by the first image range.
 2. The information processingapparatus according to claim 1, wherein the second image range includesthe inspection target.
 3. The information processing apparatus accordingto claim 1, wherein the one or more processors and the one or morenon-transitory computer-readable storage media are further configured toobtain data indicating a characteristic portion in the image, whereinthe second image range is determined based on the data.
 4. Theinformation processing apparatus according to claim 3, wherein the oneor more processors and the one or more non-transitory computer-readableare further configured to: obtain data indicating the characteristicportion within a predetermined range of distance with reference to aposition of the inspection target, and to include at least part of thecharacteristic portion in the second image range.
 5. The informationprocessing apparatus according to claim 4, wherein the one or moreprocessors and the one or more non-transitory computer-readable storageare further configured to, if there is no characteristic portion withinthe predetermined range of distance, determine a range of distance withreference to the position of the inspection target greater than thepredetermined range as the second image range.
 6. The informationprocessing apparatus according to claim 3, wherein the one or moreprocessors and the one or more non-transitory computer-readable storageare further configured to: if the data indicates a plurality ofcharacteristic portions, set a degree of priority for each of theplurality of characteristic portions, and determine the second imagerange based on at least one of the plurality of characteristic portionsselected based on the degrees of priority of the respective plurality ofcharacteristic portions.
 7. The information processing apparatusaccording to claim 6, wherein the one or more processors and the one ormore non-transitory computer-readable storage are further configured toset the degrees of priority of the respective plurality ofcharacteristic portions based on at least any one of a positionalrelationship between the inspection target and each of the plurality ofcharacteristic portions, rarity of each of the plurality ofcharacteristic portions, and conspicuity of each of the plurality ofcharacteristic portions.
 8. The information processing apparatusaccording to claim 1, wherein the one or more processors and the one ormore non-transitory computer-readable storage are further configured todetermine the first image range based on an image condition of an imageused to train a trained model to be used in making the determination. 9.The information processing apparatus according to claim 1, wherein theone or more processors and the one or more non-transitorycomputer-readable storage are further configured to select one of aplurality of trained models based on a size of the first image range,and make the determination by using the trained model selected.
 10. Theinformation processing apparatus according to claim 1, wherein the oneor more processors and the one or more non-transitory computer-readablestorage are further configured to determine the second image range tonot exceed a maximum size based on at least any one of a size of theinspection target, resolution of the image, and an image size of therecording.
 11. The information processing apparatus according to claim1, wherein the one or more processors and the one or more non-transitorycomputer-readable storage are further configured to detect an objectfrom the image, wherein the inspection target is based on a detectionresult of the detection .
 12. The information processing apparatusaccording to claim 11, wherein the inspection target is a groupincluding a first object detected by according to the detection resultand a second object detected according to the detection result, thesecond object being an object lying within a predetermined range ofdistance with reference to a position of the first object.
 13. Theinformation processing apparatus according to claim 1, wherein the oneor more processors and the one or more non-transitory computer-readablestorage are further is configured to determine a plurality of imageranges of different sizes, and determine one of the plurality of imageranges selected based on a user operation as the second image range. 14.The information processing apparatus according to claim 1, wherein theone or more processors and the one or more non-transitorycomputer-readable storage are further configured to control display of aresult of the determination using the first image range and an image cutout to the second image range.
 15. The information processing apparatusaccording to claim 1, wherein the one or more processors and the one ormore non-transitory computer-readable storage media are furtherconfigured to correct either a result of the determination using thefirst image range or the second image range based on a user operation.16. The information processing apparatus according to claim 1, whereinthe one or more processors and the one or more non-transitorycomputer-readable storage media are further configured to store a resultof the determination using the first image range and an image cut out tothe second image range in association with each other.
 17. Theinformation processing apparatus according to claim 1, wherein the oneor more processors and the one or more non-transitory computer-readablestorage media are further configured to: determine the first image rangein the image, and determine the second image range in another imagedifferent from the image.
 18. The information processing apparatusaccording to claim 1, wherein the image is a viewpoint-converted image.19. The information processing apparatus according to claim 1, whereinthe inspection target is a grid pattern crack, a hexagonal patterncrack, or a closed crack.
 20. The information processing apparatusaccording to claim 1, wherein the determination is determination of adamage degree of the inspection target.
 21. The information processingapparatus according to claim 1, wherein the one or more processors andthe one or more non-transitory computer-readable storage are furtherconfigured to output coordinate information about each of the first andsecond image ranges.
 22. An information processing apparatus comprising:one or more computer-readable storage media; and one or more processorsthat are in communication with the one or more non-transitorycomputer-readable storage media, wherein the one or more processors andthe one or more non-transitory computer-readable storage media areconfigured to: obtain an image; detect a landmark from the image, thelandmark being a characteristic portion having a characteristicdifferent from a characteristic of an area nearby; and determine animage range including an object included in the image and the landmarkas an image range to be used in recording an inspection result of aninspection target, the object being the inspection target.
 23. Aninformation processing method comprising: determining a first imagerange to be used in making a determination related to inspection of aninspection target included in an image, based on a detection result ofthe inspection target from the image; and determining a second imagerange to be used in recording an inspection result of the inspectiontarget, the second image range being an image range indicating a widerrange than a range indicated by the first image range.
 24. Anon-transitory computer readable storage medium storing a program forcausing a computer to execute an information processing method, theinformation processing method comprising: determining a first imagerange to be used in making a determination related to inspection of aninspection target included in an image, based on a detection result ofthe inspection target from the image; and determining a second imagerange to be used in recording an inspection result of the inspectiontarget, the second image range being an image range indicating a widerrange than a range indicated by the first image range.
 25. Theinformation processing apparatus according to claim 1, wherein theinspection target is on an object, and wherein the second image rangeincludes information related to a position of the inspection target onthe object.
 26. The information processing apparatus according to claim25, wherein the object is a structure, wherein the inspection target isone or more cracks, and wherein the determination related to theinspection of the inspection target is a determination of degrees ofdamage of the structure.